Strong intra smoothing for in rext

ABSTRACT

Based on conditions or restrictions, Strong Intra Smoothing (SIS) is enabled or disabled. SIS is not allowed for chroma when the chroma format is 444, and SIS is not allowed for chroma when the chroma format is 422 or 444.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims priority under 35 U.S.C. §119(e) of the U.S.Provisional Patent Application Ser. No. 61/947,207, filed Mar. 3, 2014and titled, “Strong Intra Smoothing for in RExt,” U.S. ProvisionalPatent Application Ser. No. 61/954,248, filed Mar. 17, 2014 and titled,“Strong Intra Smoothing for in RExt,” and U.S. Provisional PatentApplication Ser. No. 61/971,140, filed Mar. 27, 2014 and titled, “StrongIntra Smoothing for in RExt,” which are hereby incorporated by referencein their entireties for all purposes.

FIELD OF THE INVENTION

The present invention relates to video coding. More specifically, thepresent invention relates to high efficiency video coding.

BACKGROUND OF THE INVENTION

In JCTVC-P1005_v1, Strong Intra Smoothing (SIS) is allowed to apply tochroma when chroma format is 4:4:4. When intra smoothing disabled flagis equal to 0 and either cldx is equal to 0 or ChromaArrayType is equalto 3, the filtering process of neighboring samples specified is invokedwith the sample array p and the transform block size nTbS as inputs, andthe output is reassigned to the sample array p. In RExt6.0 SIS is onlyallowed for luma. Thus, there is a mismatch on how to deal with SIS forthe chroma in 444 chroma format.

SUMMARY OF THE INVENTION

Based on conditions or restrictions, Strong Intra Smoothing (SIS) isenabled or disabled. SIS is not allowed for chroma when the chromaformat is 444, and SIS is not allowed for chroma when the chroma formatis 422 or 444.

In one aspect, a method programmed in a non-transitory memory of adevice comprises acquiring video content and encoding the video contentwithout implementing strong intra smoothing depending on the chromaformat of the video content. The method further comprises determiningthe chroma format is 444; and not enabling strong intra smoothing. Themethod further comprises determining the chroma format is 422 or 444;and not enabling strong intra smoothing. A strong intra smoothing flagis equal to 1 and a image component index is equal to 0. The device isselected from the group consisting of a personal computer, a laptopcomputer, a computer workstation, a server, a mainframe computer, ahandheld computer, a personal digital assistant, a cellular/mobiletelephone, a smart appliance, a gaming console, a digital camera, adigital camcorder, a camera phone, a smart phone, a portable musicplayer, a tablet computer, a mobile device, a video player, a video discwriter/player, a television, a home entertainment system and smartjewelry.

In another aspect, a system programmed in a non-transitory memory of adevice comprises a receiving device configured for receiving encodedvideo content, wherein the encoded video content was encoded withoutusing strong intra smoothing depending on the chroma format of the videocontent and a display device configured for displaying the encoded videocontent. Encoding the encoded video content included determining thechroma format is 444; and not enabling strong intra smoothing. Encodingthe encoded video content included determining the chroma format is 422or 444 and not enabling strong intra smoothing. A strong intra smoothingflag is equal to 1 and a image component index is equal to 0. The deviceis selected from the group consisting of a personal computer, a laptopcomputer, a computer workstation, a server, a mainframe computer, ahandheld computer, a personal digital assistant, a cellular/mobiletelephone, a smart appliance, a gaming console, a digital camera, adigital camcorder, a camera phone, a smart phone, a portable musicplayer, a tablet computer, a mobile device, a video player, a video discwriter/player, a television, a home entertainment system and smartjewelry.

In another aspect, an apparatus comprises a display device, anon-transitory memory for storing an application, the application forreceiving video content, wherein the video content was encoded withoutusing strong intra smoothing depending on the chroma format of the videocontent and displaying the video content on the display device and aprocessing component coupled to the memory, the processing componentconfigured for processing the application. Encoding the encoded videocontent included determining the chroma format is 444; and not enablingstrong intra smoothing. Encoding the encoded video content includeddetermining the chroma format is 422 or 444; and not enabling strongintra smoothing. A strong intra smoothing flag is equal to 1 and a imagecomponent index is equal to 0. The apparatus is selected from the groupconsisting of a personal computer, a laptop computer, a computerworkstation, a server, a mainframe computer, a handheld computer, apersonal digital assistant, a cellular/mobile telephone, a smartappliance, a gaming console, a digital camera, a digital camcorder, acamera phone, a smart phone, a portable music player, a tablet computer,a mobile device, a video player, a video disc writer/player, atelevision, a home entertainment system and smart jewelry.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a flowchart of an SIS decision method according tosome embodiments.

FIG. 2 illustrates a block diagram of an exemplary computing deviceconfigured to implement the SIS decision method according to someembodiments.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

In High Efficiency Video Coding (HEVC) version 1, strong intra smoothing(SIS) is applied to luma (420 format) to reduce (e.g. smooth) somevisual artifacts in intra prediction. In Range Extension (RExt) of HEVC,more chroma formats (monochrome, 4:2:2 and 4:4:4) and higher bitdepths(12, 14, 16) are supported. Implementations to extend the SIS in RExt toaddress SIS for 444 chroma format. Restrictions on applying SIS are alsodescribed.

SIS introduces blocking artifacts and coding efficiency loss althoughSIS improves contouring artifacts. If the disabling SIS conditions arenot satisfied, then the appropriate flag is set to a value other than 0such as 1.

Filtering Process of Neighboring Samples

When filterFlag is equal to 1, the following applies: The variable bilntFlag is derived as follows:

If all of the following conditions are true, bilntFlag is set equal to1:

-   -   strong intra smoothing enabled flag is equal to 1    -   nTbS is equal to 32    -   cldx is equal to 0 //cldx is the image component index    -   Abs(p[−1][−1]+p[nTbS*2−1][−1]−2*p[nTbS−1][−1]<(1<<(BitDepth_(Y)−5))    -   Abs(p[−1][−1]+p[−1][nTbS*2−1]−2*p[−1][nTbS−1])<(1<<(BitDepth_(Y)−5))    -   Otherwise, bilntFlag is set equal to 0.

Shown herein is a comparison between SIS “on” and “off” for chroma,based on both AHG5 and AHG8 test sequences. The reference software usedis HM13.0+RExt6.0.

More specifically, SIS “on” means that SIS is allowed for chroma whenthe chroma format is 4:4:4 and SIS is applied becauseintra_smoothing_disabled_flag is set to 0 (false) in default settings.Anchor: SIS “off” is exactly the HM13.0+RExt6.0.

In the simulation, AHG5 test sequences and conditions apply. Because thesimulation is run in a cluster, encoding and decoding times are notaccurate. The results are summarized in Table 1. Results show thatallowing SIS for chroma when chroma format is 4:4:4 will degrade thecoding performance of current reference software under common testconditions.

TABLE 1 Summary of results for AHG5 conditions All Intra Main-tier AllIntra High-tier All Intra Super High-tier Y U V Y U V Y U V RGB: 4:4:40.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% YCbCr 4:4:4 0.4% 0.4% 0.4%0.2% 0.2% 0.2% 0.1% 0.1% 0.0% YCbCr 4:2:2 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%0.0% 0.0% 0.0% Enc Time[%]  99% 100% 100% Dec Time[%] 101% 101%  99%Random Access Main-tier Random Access High-tier Y U V Y U V RGB: 4:4:40.0% 0.0% 0.0% 0.0% 0.0% 0.0% YCbCr 4:4:4 0.2% 0.3% 0.1% 0.1% 0.1% 0.0%YCbCr 4:2:2 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% Enc Time[%] 100% 101% DecTime[%]  99% 101% Low delay B Main-tier Low Delay B High-tier Y U V Y UV RGB: 4:4:4 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% YCbCr 4:4:4 0.1% 0.4% 0.1%0.1% 0.1% 0.0% YCbCr 4:2:2 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% Enc Time[%]103% 101% Dec Time[%] 100%  99%

In the simulation, AHG8 test sequences and conditions apply. Because thesimulation is run in a cluster, encoding and decoding times are notaccurate. The results are summarized in Table 2. Results show thatallowing SIS for chroma when the chroma format is 4:4:4 will degrade thecoding performance of current reference software under common testconditions.

TABLE 2 Summary of results for AHG8 conditions All Intra Main-tier AllIntra High-tier All Intra Super High-tier Y U V Y U V Y U V Class F 0.0%0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% Class B 0.0% 0.0% 0.0% 0.0% 0.0%0.0% 0.0% 0.0% 0.0% RGB: 4:4:4 SC 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%0.0% 0.0% RGB: 4:4:4 Ani 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%YCbCr 4:4:4 SC 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% YCbCr 4:4:4Ani 0.2% 0.2% 0.2% 0.1% 0.1% 0.1% 0.1% 0.0% 0.0% RangExt 0.2% 0.2% 0.1%0.1% 0.1% 0.0% 0.0% 0.0% 0.0% RGB4:4:4SC(op) 0.0% 0.0% 0.0% 0.0% 0.0%0.0% 0.0% 0.0% 0.0% YCbCr 4:4:4 SC (optional) 0.0% 0.0% 0.0% 0.0% 0.0%0.0% 0.0% 0.0% 0.0% Enc Time[%] Dec Time[%] Random Access Main-tierRandom Access High-tier Y U V Y U V Class F 0.0% 0.0% 0.0% 0.0% 0.0%0.0% Class B 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% RGB: 4:4:4 SC 0.0% 0.1% 0.0%0.1% 0.1% 0.1% RGB: 4:4:4 Anim 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% YCbCr 4:4:4SC 0.1% 0.1% 0.1% 0.1% 0.1% 0.0% YCbCr 4:4:4 Anim 0.1% 0.0% 0.0% 0.1%0.0% 0.0% RangExt 0.1% 0.0% 0.0% 0.0% 0.0% 0.0% RGB4:4:4 SC (optional)0.1% 0.1% 0.2% 0.0% 0.0% 0.0% YCbCr 4:4:4 SC (optional) 0.0% 0.0% 0.0%0.0% −0.1% 0.0% Enc Time[%] Dec Time[%] Low delay B Main-tier Low DelayB High-tier Y U V Y U V Class F 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% Class B0.0% 0.0% 0.0% 0.0% 0.0% 0.0% RGB: 4:4:4 SC 0.1% 0.0% 0.1% 0.1% 0.0%0.1% RGB: 4:4:4 Anim −0.1% −0.1% 0.0% −0.1% 0.0% 0.0% YCbCr 4:4:4 SC−0.1% 0.0% −0.1% −0.1% 0.0% −0.2% YCbCr 4:4:4 Anim 0.0% 0.0% 0.1% 0.0%0.0% 0.1% RangExt 0.0% 0.1% 0.0% 0.0% 0.0% 0.0% RGB4:4:4 SC (optional)−0.7% −0.8% −0.7% −0.2% −0.3% −0.3% YCbCr 4:4:4 SC (optional) −0.6%−0.7% −0.5% −0.7% −0.8% −0.7% Enc Time[%] Dec Time[%]

FIG. 1 illustrates a flowchart of an SIS decision method according tosome embodiments. In the step 100, video content is acquired,transmitted and/or received. In the step 102, the video content isencoded. The video content is encoded using SIS with theconditions/restrictions described herein. For example, SIS is notallowed (e.g., disabled) for chroma when the chroma format is 444. Inanother example, SIS is not allowed for chroma when the chroma format is422 or 444. In the step 104, the video content is displayed. In someembodiments, fewer or additional steps are implemented. For example, theencoded video content is decoded after it is encoded. In someembodiments, the order of the steps is modified. For example, the videocontent is encoded and then transmitted.

FIG. 2 illustrates a block diagram of an exemplary computing deviceconfigured to implement the SIS decision method according to someembodiments. The computing device 200 is able to be used to acquire,store, compute, process, communicate and/or display information such asimages and videos. In general, a hardware structure suitable forimplementing the computing device 200 includes a network interface 202,a memory 204, a processor 206, I/O device(s) 208, a bus 210 and astorage device 212. The choice of processor is not critical as long as asuitable processor with sufficient speed is chosen. The memory 204 isable to be any conventional computer memory known in the art. Thestorage device 212 is able to include a hard drive, CDROM, CDRW, DVD,DVDRW, High Definition disc/drive, ultra-HD drive, flash memory card orany other storage device. The computing device 200 is able to includeone or more network interfaces 202. An example of a network interfaceincludes a network card connected to an Ethernet or other type of LAN.The I/O device(s) 208 are able to include one or more of the following:keyboard, mouse, monitor, screen, printer, modem, touchscreen, buttoninterface and other devices. SIS decision method application(s) 230 usedto implement the SIS decision method are likely to be stored in thestorage device 212 and memory 204 and processed as applications aretypically processed. More or fewer components shown in FIG. 2 are ableto be included in the computing device 200. In some embodiments, SISdecision method hardware 220 is included. Although the computing device200 in FIG. 2 includes applications 230 and hardware 220 for the SISdecision method, the SIS decision method is able to be implemented on acomputing device in hardware, firmware, software or any combinationthereof. For example, in some embodiments, the SIS decision methodapplications 230 are programmed in a memory and executed using aprocessor. In another example, in some embodiments, the SIS decisionmethod hardware 220 is programmed hardware logic including gatesspecifically designed to implement the SIS decision method.

In some embodiments, the SIS decision method application(s) 230 includeseveral applications and/or modules. In some embodiments, modulesinclude one or more sub-modules as well. In some embodiments, fewer oradditional modules are able to be included.

Examples of suitable computing devices include a personal computer, alaptop computer, a computer workstation, a server, a mainframe computer,a handheld computer, a personal digital assistant, a cellular/mobiletelephone, a smart appliance, a gaming console, a digital camera, adigital camcorder, a camera phone, a smart phone, a portable musicplayer, a tablet computer, a mobile device, a video player, a video discwriter/player (e.g., DVD writer/player, high definition discwriter/player, ultra high definition disc writer/player), a television,a home entertainment system, smart jewelry (e.g., smart watch) or anyother suitable computing device.

To utilize the SIS decision method described herein, a device such as adigital camera/camcorder is used to acquire image/video content or adevice such as a television is used to display video content. The SISdecision method is automatically used during the encoding process of theacquired or displayed video content such as when the video content istransmitted. The SIS decision method is able to be implemented with userassistance or automatically without user involvement.

In operation, the SIS decision method determines when SIS is implementedto ensure proper and efficient encoding of content.

Some Embodiments of Strong Intra Smoothing for In Rext

-   -   1. A method programmed in a non-transitory memory of a device        comprising:        -   a. acquiring video content; and        -   b. encoding the video content without implementing strong            intra smoothing depending on the chroma format of the video            content.    -   2. The method of clause 1 further comprising determining the        chroma format is 444; and not enabling strong intra smoothing.    -   3. The method of clause 1 further comprising determining the        chroma format is 422 or 444; and not enabling strong intra        smoothing.    -   4. The method of clause 1 wherein a strong intra smoothing flag        is equal to 1 and a image component index is equal to 0.    -   5. The method of clause 1 wherein the device is selected from        the group consisting of a personal computer, a laptop computer,        a computer workstation, a server, a mainframe computer, a        handheld computer, a personal digital assistant, a        cellular/mobile telephone, a smart appliance, a gaming console,        a digital camera, a digital camcorder, a camera phone, a smart        phone, a portable music player, a tablet computer, a mobile        device, a video player, a video disc writer/player, a        television, a home entertainment system and smart jewelry.    -   6. A system programmed in a non-transitory memory of a device        comprising:        -   a. a receiving device configured for receiving encoded video            content, wherein the encoded video content was encoded            without using strong intra smoothing depending on the chroma            format of the video content; and        -   b. a display device configured for displaying the encoded            video content.    -   7. The system of clause 6 wherein encoding the encoded video        content included determining the chroma format is 444; and not        enabling strong intra smoothing.    -   8. The system of clause 6 wherein encoding the encoded video        content included determining the chroma format is 422 or 444;        and not enabling strong intra smoothing.    -   9. The system of clause 6 wherein a strong intra smoothing flag        is equal to 1 and a image component index is equal to 0.    -   10. The system of clause 6 wherein the device is selected from        the group consisting of a personal computer, a laptop computer,        a computer workstation, a server, a mainframe computer, a        handheld computer, a personal digital assistant, a        cellular/mobile telephone, a smart appliance, a gaming console,        a digital camera, a digital camcorder, a camera phone, a smart        phone, a portable music player, a tablet computer, a mobile        device, a video player, a video disc writer/player, a        television, a home entertainment system and smart jewelry.    -   11. An apparatus comprising:        -   a. a display device;        -   b. a non-transitory memory for storing an application, the            application for:            -   i. receiving video content, wherein the video content                was encoded without using strong intra smoothing                depending on the chroma format of the video content; and            -   ii. displaying the video content on the display device;                and        -   c. a processing component coupled to the memory, the            processing component configured for processing the            application.    -   12. The apparatus of clause 11 wherein encoding the encoded        video content included determining the chroma format is 444; and        not enabling strong intra smoothing.    -   13. The apparatus of clause 11 encoding the encoded video        content included determining the chroma format is 422 or 444;        and not enabling strong intra smoothing.    -   14. The apparatus of clause 11 wherein a strong intra smoothing        flag is equal to 1 and a image component index is equal to 0.    -   15. The apparatus of clause 11 wherein the apparatus is selected        from the group consisting of a personal computer, a laptop        computer, a computer workstation, a server, a mainframe        computer, a handheld computer, a personal digital assistant, a        cellular/mobile telephone, a smart appliance, a gaming console,        a digital camera, a digital camcorder, a camera phone, a smart        phone, a portable music player, a tablet computer, a mobile        device, a video player, a video disc writer/player, a        television, a home entertainment system and smart jewelry.

The present invention has been described in terms of specificembodiments incorporating details to facilitate the understanding ofprinciples of construction and operation of the invention. Suchreference herein to specific embodiments and details thereof is notintended to limit the scope of the claims appended hereto. It will bereadily apparent to one skilled in the art that other variousmodifications may be made in the embodiment chosen for illustrationwithout departing from the spirit and scope of the invention as definedby the claims.

What is claimed is:
 1. A method programmed in a non-transitory memory ofa device comprising: a. acquiring video content; and b. encoding thevideo content without implementing strong intra smoothing depending onthe chroma format of the video content.
 2. The method of claim 1 furthercomprising determining the chroma format is 444; and not enabling strongintra smoothing.
 3. The method of claim 1 further comprising determiningthe chroma format is 422 or 444; and not enabling strong intrasmoothing.
 4. The method of claim 1 wherein a strong intra smoothingflag is equal to 1 and a image component index is equal to
 0. 5. Themethod of claim 1 wherein the device is selected from the groupconsisting of a personal computer, a laptop computer, a computerworkstation, a server, a mainframe computer, a handheld computer, apersonal digital assistant, a cellular/mobile telephone, a smartappliance, a gaming console, a digital camera, a digital camcorder, acamera phone, a smart phone, a portable music player, a tablet computer,a mobile device, a video player, a video disc writer/player, atelevision, a home entertainment system and smart jewelry.
 6. A systemprogrammed in a non-transitory memory of a device comprising: a. areceiving device configured for receiving encoded video content, whereinthe encoded video content was encoded without using strong intrasmoothing depending on the chroma format of the video content; and b. adisplay device configured for displaying the encoded video content. 7.The system of claim 6 wherein encoding the encoded video contentincluded determining the chroma format is 444; and not enabling strongintra smoothing.
 8. The system of claim 6 wherein encoding the encodedvideo content included determining the chroma format is 422 or 444; andnot enabling strong intra smoothing.
 9. The system of claim 6 wherein astrong intra smoothing flag is equal to 1 and a image component index isequal to
 0. 10. The system of claim 6 wherein the device is selectedfrom the group consisting of a personal computer, a laptop computer, acomputer workstation, a server, a mainframe computer, a handheldcomputer, a personal digital assistant, a cellular/mobile telephone, asmart appliance, a gaming console, a digital camera, a digitalcamcorder, a camera phone, a smart phone, a portable music player, atablet computer, a mobile device, a video player, a video discwriter/player, a television, a home entertainment system and smartjewelry.
 11. An apparatus comprising: a. a display device; b. anon-transitory memory for storing an application, the application for:i. receiving video content, wherein the video content was encodedwithout using strong intra smoothing depending on the chroma format ofthe video content; and ii. displaying the video content on the displaydevice; and c. a processing component coupled to the memory, theprocessing component configured for processing the application.
 12. Theapparatus of claim 11 wherein encoding the encoded video contentincluded determining the chroma format is 444; and not enabling strongintra smoothing.
 13. The apparatus of claim 11 encoding the encodedvideo content included determining the chroma format is 422 or 444; andnot enabling strong intra smoothing.
 14. The apparatus of claim 11wherein a strong intra smoothing flag is equal to 1 and a imagecomponent index is equal to
 0. 15. The apparatus of claim 11 wherein theapparatus is selected from the group consisting of a personal computer,a laptop computer, a computer workstation, a server, a mainframecomputer, a handheld computer, a personal digital assistant, acellular/mobile telephone, a smart appliance, a gaming console, adigital camera, a digital camcorder, a camera phone, a smart phone, aportable music player, a tablet computer, a mobile device, a videoplayer, a video disc writer/player, a television, a home entertainmentsystem and smart jewelry.